Robust classification of high dimensional unbalanced single and multi-label datasets

University of Technology Sydney. Faculty of Engineering and Information Technology.Single and multi-label classification are arguably two of the most important topics within the field of machine learning. Single-label classification refers to the case where each sample is assigned to one class, and multi-label classification is where instances are associated with multiple labels simultaneously. Nowadays, research to build robust single and multi-label classification models is still ongoing in the data analytics community because of the emerging complexities in the real-world data, and due to the increasingly research interest in use of data analytics techniques in many fields including biomedicine, finance, text mining, text categorization, and images. Real-world datasets contain complexities which degrade the performance of classifiers. These complexities or open challenges are: imbalanced data, low numbers of samples, high-dimensionality, highly correlated features, label correlations, and missing labels in multi-label space. Several research gaps are identified and motivate this thesis. Class imbalance occurs when the distribution of classes is not uniform among samples. Feature extraction is used to reduce the dimensionality of data. However, the presence of highly imbalanced data in single-label classification misleads existing unsupervised and supervised feature extraction techniques. It produces features biased towards classification of the class with the majority of samples, and results in poor classification performance especially for the minor class. Furthermore, imbalanced multi-labeled data is more ubiquitous than single-labeled data because of several issues including label correlation, incomplete multi-label matrices, and noisy and irrelevant features. High-dimensional highly correlated data exist in several domains such as genomics. Many feature selection techniques consider correlated features as redundant and therefore need to be removed. Several studies investigate the interpretation of the correlated features in domains such as genomics, but investigating the classification capabilities of the correlated feature groups in single-labeled data is a point of interest in several domains. Moreover, high-dimensional multi-labeled data is more challenging than single-labeled data. Only relatively few feature selection methods have been proposed to select the discriminative features among multiple labels due to issues including interdependent labels, different instances sharing different label correlations, correlated features, and missing and noisy labels. This thesis proposes a series of novel algorithms for machine learning to handle the negative effects of the above mentioned problems and improves the performance of the classifiers in single and multi-labeled data. There are seven contributions in this thesis. Contribution 1 proposes novel cost-sensitive principal component analysis (CSPCA) and cost-sensitive non-negative matrix factorization (CSNMF) methods for handling feature extraction of imbalanced single-labeled data. Contribution 2 extends a standard non-negative matrix factorization to a balanced supervised non-negative matrix factorization (BSNMF) to handle the class imbalance problem in supervised non-negative matrix factorization. Contribution 3 introduces an ABC-Sampling algorithm for balancing imbalanced datasets based on Artificial Bee Colony algorithm. Contribution 4 develops a novel supervised feature selection algorithm (SCANMF) by jointly integrating correlation network and structural analysis of the balanced supervised non-negative matrix factorization to handle high-dimensional, highly correlated single-labeled data. Contribution 5 proposes an ensemble feature ranking method using co-expression networks to select optimal features for classification. Contribution 6 proposes a Correlated- and Multi-label Feature Selection method (CMFS), based on NMF for simultaneously performing multi-label feature selection and addressing the following challenges: interdependent labels, different instances sharing different label correlations, correlated features, and missing and awed labels. Contribution 7 presents an integrated multi-label approach (ML-CIB) for simultaneously training the multi-label classification model and addressing the following challenges namely, class imbalance, label correlation, incomplete multi-label matrices, and noisy and irrelevant features. The performance of all novel algorithms in this thesis is evaluated in terms of single and multi-label classification accuracy. The proposed algorithms are evaluated in the context of a childhood leukaemia dataset from The Children Hospital at Westmead, and public datasets for different fields including genomics, finance, text mining, images, and others from online repositories. Moreover, all the results of the proposed algorithms in this thesis are compared to state-of-the-art methods. The experimental results indicate that the proposed algorithms outperform the state-of-the-art methods. Further, several statistical tests including, t-test and Friedman test are applied to evaluate the results to demonstrate the statistical significance of the proposed methods in this thesis

SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentation

This paper proposes a novel self-supervised based Cut-and-Paste GAN to perform foreground object segmentation and generate realistic composite images without manual annotations. We accomplish this goal by a simple yet effective self-supervised approach coupled with the U-Net based discriminator. The proposed method extends the ability of the standard discriminators to learn not only the global data representations via classification (real/fake) but also learn semantic and structural information through pseudo labels created using the self-supervised task. The proposed method empowers the generator to create meaningful masks by forcing it to learn informative per-pixel as well as global image feedback from the discriminator. Our experiments demonstrate that our proposed method significantly outperforms the state-of-the-art methods on the standard benchmark datasets

Optimization of a Small Wind Turbine for a Rural Area: A Case Study of Deniliquin, New South Wales, Australia

The performance of a wind turbine is affected by wind conditions and blade shape. This study aimed to optimize the performance of a 20 kW horizontal-axis wind turbine (HAWT) under local wind conditions at Deniliquin, New South Wales, Australia. Ansys Fluent (version 18.2, Canonsburg, PA, USA) was used to investigate the aerodynamic performance of the HAWT. The effects of four Reynolds-averaged Navier–Stokes turbulence models on predicting the flows under separation condition were examined. The transition SST model had the best agreement with the NREL CER data. Then, the aerodynamic shape of the rotor was optimized to maximize the annual energy production (AEP) in the Deniliquin region. Statistical wind analysis was applied to define the Weibull function and scale parameters which were 2.096 and 5.042 m/s, respectively. The HARP_Opt (National Renewable Energy Laboratory, Golden, CO, USA) was enhanced with design variables concerning the shape of the blade, rated rotational speed, and pitch angle. The pitch angle remained at 0° while the rising wind speed improved rotor speed to 148.4482 rpm at rated speed. This optimization improved the AEP rate by 9.068% when compared to the original NREL design

Study of Pressure Retarded Osmosis Process in Hollow Fiber Membrane: Cylindrical Model for Description of Energy Production

A new mathematical model was developed to predict the cylindrical effect of the membrane performance in the pressure retarded osmosis process. The cylindrical membrane transport layers (the draw side boundary and the porous membrane) were divided into very thin sublayers with constant mass transport parameters, among others with a constant radius in every sublayer. The obtained second-order differential mass balance equations were solved analytically, with constant parameters written for every sublayer. The algebraic equation system involving 2N equations was then solved for the determinant solution. It was shown that the membrane properties, water permeability (A), salt permeability (B), structural parameter (S) and the operating conditions (inlet draw side solute concentration and draw side mass transfer coefficient) affect the water flux strongly, and thus the membrane performance, due to the cylindrical effect caused by the variable surface and volume of the sublayers. This effect significantly depends on the lumen radius. The lower radius means a larger change in the internal surface/volume of sublayers with ΔR thickness. The predicted results correspond to that of the flat-sheet membrane layer at ro = 10,000 μm. At the end of this manuscript, the calculated mass transfer rates were compared to those measured. It was stated that the curvature effect in using a capillary membrane must not be left out of consideration when applying hollow fiber membrane modules due to their relatively low lumen radius. The presented model provides more precise prediction of the performance in the case of hollow fiber membranes

Study of Pressure Retarded Osmosis Process in Hollow Fiber Membrane: Cylindrical Model for Description of Energy Production

A new mathematical model was developed to predict the cylindrical effect of the membrane performance in the pressure retarded osmosis process. The cylindrical membrane transport layers (the draw side boundary and the porous membrane) were divided into very thin sublayers with constant mass transport parameters, among others with a constant radius in every sublayer. The obtained second-order differential mass balance equations were solved analytically, with constant parameters written for every sublayer. The algebraic equation system involving 2N equations was then solved for the determinant solution. It was shown that the membrane properties, water permeability (A), salt permeability (B), structural parameter (S) and the operating conditions (inlet draw side solute concentration and draw side mass transfer coefficient) affect the water flux strongly, and thus the membrane performance, due to the cylindrical effect caused by the variable surface and volume of the sublayers. This effect significantly depends on the lumen radius. The lower radius means a larger change in the internal surface/volume of sublayers with ΔR thickness. The predicted results correspond to that of the flat-sheet membrane layer at ro = 10,000 μm. At the end of this manuscript, the calculated mass transfer rates were compared to those measured. It was stated that the curvature effect in using a capillary membrane must not be left out of consideration when applying hollow fiber membrane modules due to their relatively low lumen radius. The presented model provides more precise prediction of the performance in the case of hollow fiber membranes

SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentation

This paper proposes a novel self-supervised based Cut-and-Paste GAN to perform foreground object segmentation and generate realistic composite images without manual annotations. We accomplish this goal by a simple yet effective self-supervised approach coupled with the U-Net discriminator. The proposed method extends the ability of the standard discriminators to learn not only the global data representations via classification (real/fake) but also learn semantic and structural information through pseudo labels created using the self-supervised task. The proposed method empowers the generator to create meaningful masks by forcing it to learn informative per-pixel and global image feedback from the discriminator. Our experiments demonstrate that our proposed method significantly outperforms the state-of-the-art methods on the standard benchmark datasets

Box plot of AUC accuracies on the four microarray datasets.

<p>Comparison of AUC accuracies on the four microarray datasets estimated using the 0.632+ bootstrap method with 100 bootstrap samples. (a) Colon dataset, (b) Breast2 dataset, (c) Breast3 dataset, and (d) NCI dataset.</p

ESVM-RFE vs SVM-RFE.

<p>Classification performance comparison between ESVM-RFE and SVM-RFE estimated using the 0.632+ bootstrap method with 100 bootstrap samples at number of features equal to 36 and 46, respectively.</p

Comparison of ESVM-RFE, SVM-RFE and BIRF in terms of time.

<p>Comparison of ESVM-RFE, SVM-RFE and BIRF in terms of time.</p

Microarray gene expression datasets.

<p>Microarray gene expression datasets.</p